Flowbased dynamic load balancing for passive network monitoring

Cluster based packet capturing is a way of overcoming the speed of a slow disk to tap a high-speed network. Most cluster-based architectures, however, do not consider load balancing as an important issue. In order to perform monitoring at full line speed without losing packets, we accept that the balance among back-end servers must be maintained. Conventional methods rely on fixed or random routing where a ”source-destination” address pair is used as a unit for load balancing. Instead of using such a coarse grain unit, we use a flow, a ”source-destination-port” pair. In addition, we adaptively balance loads of back-end servers using a flow load estimation technique. The proposed methods have been validated by performing a trace-based simulation. Compared to the existing routing approaches, our method balances load of back-end servers. From a simulation, we find that for protocols having a transient load with a small variance in flow length, we may use simple methods of load balancing such as round-robin, but for those having a persistent load with a large variance, we need to use a flow estimation technique.